make gener classifi robust select bia andrew smith charl elkan paper present approach semi supervis learn label train data test data differ distribut specif sampl select label bias subset gener distribut test set consist sampl drawn gener distribut distribut unlabel sampl appear loan applic approv sampl repai default label exist approv applic goal model repai default behavior applic appear spam filter label sampl date due cost label email hand unlabel set date email exist goal build filter sort incom email approach overcom bia literatur reli assumpt sampl select label depend featur label case provabl correct method exist miss label miss random mar real applic select bia sever mar condit independ assumpt satisfi miss label miss random mnar learn method provabl correct present gener classifi shift mixtur model smm separ represent distribut label sampl unlabel sampl smm make condit independ assumpt model distribut semi label data set arbitrari bia label present learn method base expect maxim em algorithm overcom arbitrari label bia learn smm higher test set accuraci real world data set mnar bia exist learn method proven overcom mar bia 